Skip to content

feat: Add 5 additional external system connectors (CFOS-26)#53

Merged
jayscambler merged 1 commit intomainfrom
jay/cfos-26-external-system-connectors
Jun 25, 2025
Merged

feat: Add 5 additional external system connectors (CFOS-26)#53
jayscambler merged 1 commit intomainfrom
jay/cfos-26-external-system-connectors

Conversation

@jayscambler
Copy link
Copy Markdown
Contributor

Summary

This PR adds 5 additional external system connectors to complement the GitHub and Linear connectors already merged in #52:

  • Google Drive Connector
  • Notion Connector
  • Slack Connector
  • Discord Connector
  • Obsidian Connector

Implementation Details

New Connectors

  1. Google Drive Connector

    • Import documents from Google Drive (Docs, Sheets, Slides)
    • Export Google formats to readable text
    • Support for folders and shared drives
    • OAuth and service account authentication
  2. Notion Connector

    • Import pages, databases, and database entries
    • Convert Notion blocks to markdown
    • Preserve page hierarchy and relationships
    • Extract properties and metadata
  3. Slack Connector

    • Import messages and threads from channels
    • Preserve thread replies and reactions
    • Support for user mentions and attachments
    • Configurable time windows for sync
  4. Discord Connector

    • Import messages from servers and channels
    • Support for threads and forum posts
    • Handle embeds and attachments
    • Async implementation with discord.py
  5. Obsidian Connector

    • Import markdown notes from local vaults
    • Extract frontmatter metadata
    • Parse and preserve wiki-style backlinks
    • Support for tags and folder structure

Testing

  • Added comprehensive test suite in test_all_connectors.py
  • All connectors tested for proper inheritance and method implementation
  • Import tests verify all connectors are accessible

Documentation

  • Updated docs/external-connectors.md with examples for all new connectors
  • Created examples/all_connectors_usage.py with comprehensive examples
  • Each connector includes detailed docstrings

Related Issues

Checklist

  • All 5 new connectors implemented
  • Tests added and passing
  • Documentation updated
  • Examples provided
  • Code linted and formatted

The connectors provide a unified way to import data from major enterprise knowledge sources into ContextFrame datasets.

- Add GoogleDriveConnector for importing Google Drive documents
  - Support for Google Docs, Sheets, Slides export to readable formats
  - Handle both personal and shared drives with folder filtering
  - OAuth and service account authentication support

- Add NotionConnector for importing Notion workspace content
  - Import pages, databases, and database entries
  - Parse rich text and blocks to markdown
  - Preserve page hierarchy and relationships
  - Support for comments and page metadata

- Add SlackConnector for importing Slack workspace messages
  - Import from specific channels or entire workspace
  - Preserve thread structure and replies
  - Include reactions, user info, and attachments
  - Support for incremental sync with configurable time windows

- Add DiscordConnector for importing Discord server content
  - Import messages from specific servers and channels
  - Support for threads, forum posts, and reactions
  - Handle embeds, attachments, and message relationships
  - Async-based implementation using discord.py

- Add ObsidianConnector for importing Obsidian vault content
  - Import markdown notes with frontmatter extraction
  - Parse and preserve wiki-style backlinks
  - Include attachments and folder structure
  - Extract tags and create relationship mappings

All connectors follow the same base architecture with:
- Incremental sync capabilities
- Collection-based organization
- Comprehensive error handling and logging
- Configurable filtering and authentication options

Updated documentation and examples to include all connectors.
Added comprehensive test suite for connector imports and functionality.
@linear
Copy link
Copy Markdown

linear bot commented Jun 25, 2025

@jayscambler jayscambler merged commit 88accba into main Jun 25, 2025
@jayscambler jayscambler deleted the jay/cfos-26-external-system-connectors branch June 25, 2025 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant